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SUMURY 


tests  of  psychonotor  ability  (hand-eye  coordination)  previously  developed  by  the  Air 
Force  Huaan  Resources  Laboratory  were  validated  against  USAF  Undergraduate  Pilot  Training  (UPT) 
performance.  The  two  tests  significantly  predicted  UPT  eliminations  and  differentiated  between 
the  better  and  weaker  graduating  students.  Various  operational  Implementation  strategies  for  the 
psychomotor  tests  were  evaluated,  Including  using  the  tests  by  themselves  as  an  additional 
selection  gate  and  combining  the  scores  from  the  tests  with  other  normally  available  selection 
Information.  The  combined  approach  proved  most  accurate,  and  three  Integrated  Pilot  Candidate 
Selection  Models  (IPCSMs)  were  developed  using  combinations  of  the  psychomotor  tests.  Air  Force 
Officer  Qualifying  Test  scores,  grades  from  the  Flight  Screening  Program  and  biographical  data. 
Comparisons  among  the  IPCSMs  were  made  and  recoonendatlons  for  Implementation  were  presented. 
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PREFACE 


Work  Mas  accomplished  In  support  of  RPR  78-11,  Selection  for  Undergraduate  Pilot 
Training,  Issued  by  USAF  Air  Training  Command.  This  work  was  performed  under  Project 
7719,  Air  Force  Personnel  System  Development  on  Selection,  Assignment,  Evaluation, 
Quality  Control,  Retention,  Promotion,  and  Utilization;  Task  771918,  Selection  and 
Classification  Instruments  for  Officer  Personnel  Programs. 

The  groundwork  for  this  report  resulted  from  the  efforts  of  many  AFHRL  scientists 
beside  those  referenced  In  the  blbllograpl^.  Dr.  Dave  Hunter  and  Major  John  Quebe  were 
Instrumental  In  the  development  of  the  psychomotor  test  equipment.  Mr.  Dick  Nicewonger 
was  principal  psychomotor  test  administrator.  Mr.  Cal  Fresne  was  responsible  for 
building  the  data  files  for  the  analyses.  These  contributions  and  the  untiring  efforts 
of  Mr.  Ed  Watkins  to  program  the  analyses  are  deeply  appreciated  by  the  authors. 

In  addition,  the  authors  extend  thanks  to  Major  E.  Simpson  (ATC/TTXPT)  for  his 
coordination  of  the  operational  guidance  and  support  within  the  Air  Training  Command. 
The  Implementation  of  psychomotor  screening  within  the  USAF  Is  significantly  related  to 
his  dedicated  efforts. 
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UTILIZATION  OF  PSYCHOMOTOR  SCREENING  FOR  USAF  PILOT  CANDIDATES: 
INDEPENDENT  AND  INTEGRATED  SELECTION  METHODOLOGIES 


1.  INTRODUCTION 

A  multi-year  research  and  development  (RjD)  program  was  designed  at  the  request  of  the  United 
States  Air  Force  (USAF)  Air  Training  Coomand  (ATC)  to  improve  procedures  for  the  selection  of 
candidates  for  USAF  Undergraduate  Pilot  Training  (UPT).  The  overall  RiD  program  Is  described  In 
Kantor  and  Bordelon  (1985).  Two  Important  objectives  of  this  research  were  to  capitalize  on 
state-of-the-art  technologies  In  computer-aided  testing  for  the  development  of  accurate  measures 
of  psychomotor  ability  (hand-eye  coordination)  and  then  to  Investigate  different  ways  to  use 
these  psychomotor  measures  In  the  selection  of  UPT  candidates. 

The  principal  goal  of  selection  for  UPT  Is  to  screen  out  those  candidates  with  extremely  low 
chances  of  completing  training.  According  to  ATC  calculations,  the  average  cost  of  each  UPT 
ellmlnee  Is  approximately  $64,000  (FY84  dollars).  Therefore,  screening  out  potential  failures 
can  avoid  considerable  attrition  costs.  Also,  the  Increased  complexity  of  the  weapon  systems  and 
missions  In  which  USAF  aircrews  are  employed  requires  selection  of  high-quality  students.  Flying 
aptitude  tests  are  traditionally  used  to  differentiate  among  UPT  candidates. 

The  Ideal  selection  system  would  measure  and  consider  all  relevant  characteristics  of  a 
candidate.  These  would  Include  physical  and  mental  characteristics  as  well  as  the  candidate's 
previous  experiences,  such  as  actual  flying.  This  Information  would  be  used  to  predict  a 
candidate's  probability  of  success  In  an  optimally  weighted  manner.  The  candidates  with  the 
lowest  probability  of  success  would  be  screened  out  before  training. 

The  UPT  selection  system  currently  used  by  the  USAF  does  not  consider  all  relevant 
Information  about  candidates.  Thus,  many  candidates  fall  training  because  of  deficiencies  in 
flying  aptitude.  Psychomotor  ability  Is  one  of  several  characteristics  that  historically  has 
demonstrated  relevance  to  flying  performance.  Though  not  currently  used  by  the  USAF,  psychomotor 
screening  is  used  by  several  other  national  Air  Forces  to  select  their  pilot  candidates.  This 
report  documents  the  validation  of  two  tests  of  psychomotor  ability  against  USAF  UPT  performance 
and  develops  strategies  to  Incorporate  this  Information  Into  the  USAF  pilot  selection  system. 


Background 

Between  February  1942  and  July  1955,  measurement  of  the  psychomotor  ability  of  pilot  training 
candidates  was  an  Important  component  of  the  Air  Force  (prior  to  1947,  the  Amy  Air  Corps)  pilot 
election  procedure.  Screening  on  psychomotor  ability  was  discontinued  in  1955  because  the 
jlectro-mechanical  equipment  used  to  measure  it  was  difficult  to  calibrate  under  decentralized 
testing  conditions  and  was  hard  to  maintain  (Passey  8  McLaurIn,  1960.  Since  1955,  the  principal 
components  considered  In  USAF  pilot  selection  have  Included  physiological  fitness,  the 
paper-and-pencll  Air  Force  Officer  Qualifying  Test  (AFOQl),  and  actual  flying  experience. 
Recently,  the  candidate's  age  and  possession  of  a  technical  degree  (defined  as  a  college  degree 
In  math,  science,  engineering,  or  computer  science)  have  been  included  as  factors  in  the 
selection  decision. 

Improvements  In  computer  technology  provided  the  potential  to  develop  reliable  and 
maintainable  psychomotor  tests  and  revived  Interest  In  psychomotor  assessment  for  pilot  candidate 
screening.  In  1969,  the  Air  Force  Human  Resources  Laboratory  (AFHRI.)  contracted  to  design, 
develop,  and  determine  the  reliability  of  two  computer-based  psychomotor  tests  (Sanders, 
Valentine,  4  McGrevy,  1971).  The  basic  task  elements  of  these  new  tests  and  the  responses 
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required  of  the  subject  duplicated  earlier  electro-mechanical  tests  developed  in  1942  by  the  Army 
Air  Forces  School  of  Aviation  Medicine  (now  the  USAF  School  of  Aerospace  Medicine).  The  newly 
designed  tests  differed  from  the  earlier  versions  in  several  respects,  but  the  most  important 
difference  was  that,  for  the  first  time,  a  minicomputer  controlled  the  test  procedures,  providing 
reliability  and  standardization  not  achievable  on  electro-mechanical  devices. 

The  first  of  these  tests,  called  Two-Hand  Coordination,  is  a  pursuit  tracking  task  measuring 
the  subject's  ability  to  coordinate  the  movement  of  both  hands  to  track  a  moving  target.  The 
second  test,  called  Complex  Coordination,  requires  multi-limb,  dual-task  compensatory  tracking 
and  measures  the  subject's  ability  to  adjust  two  independent  responses  to  control  a  changing 
stimulus  pattern.  The  effectiveness  of  the  tests  was  initially  demonstrated  with  a  sample  of  120 
Officer  Training  School  (OTS)  cadets.  Stable  task  performance  was  obtained  after  10-minute  test 
sessions  and  reliabilities  were  very  high.  Inter-test  correlation  was  low,  suggesting  that  each 
test  may  contain  unique  information  and  that  both  should  be  further  evaluated.  The  high  multiple 
correlation  between  the  scores  on  both  tests  and  training  outcome  at  DPT  (R  =  .46)  suggested  that 
these  psychomotor  tests  could  enhance  the  prediction  of  pilot  training  success.  These  results 
precipitated  a  recommendation  for  large-scale  validation  of  an  operational  version  of  the 
psychomotor  tests  (McGrevy  A  Valentine,  1974). 

In  1975,  AFHRL  contracted  with  Systems  Research  Laboratory,  Inc.,  Dayton,  Ohio,  to  design  and 
develop  five  computer-based  test  devices  that  could  administer  the  two  psychomotor  tests.  These 
transportable  devices  were  self-contained  and  did  not  require  highly  trained  test 
administrators.  The  devices  faithfully  reproduced  the  two  tests  that  BioTechnology,  Inc.  had 
developed  on  the  minicomputer. 


Objectives 

In  1978,  AFHRL  began  a  multi-year  R4D  effort  to  use  the  new  test  devices  to  obtain 
psychomotor  measures  from  USAF  pilot  candidates  from  all  three  commissioning  sources:  Air  Force 
Academy  (AFA),  Air  Force  Reserve  Officer  Training  Corps  (AFROTC),  and  Officer  Training  School 
(OTS).  Initially,  the  actual  UPT  outcomes  of  the  candidates  would  be  used  to  validate  the 
usefulness  of  the  psychomotor  scores  for  pilot  screening.  (Though  not  part  of  this  report,  these 
test  scores  will  later  be  validated  against  measures  of  successful  operational  flying.)  In 
addition,  this  study  sought  to  determine  the  best  manner  to  integrate  the  psychomotor  test 
information  with  available  selection  information  to  achieve  the  greatest  improvement  in  the  USAF 
pilot  selection  process. 


II.  METHOD 


Predictive  Validation 

A  testing  plan  was  jointly  developed  between  AFHRL,  ATC,  AFA,  and  AFROTC.  In  the  spring  of 
1978,  four  of  the  psychomotor  test  devices  were  transported  to  the  Air  Force  Academy  to  test  the 
cadets  from  the  three  upper  classes.  In  the  spring  of  1979,  the  devices  were  shuttled  by  van  to 
all  of  the  AFROTC  detachments  in  the  continental  United  States  to  test  AFROTC  juniors  and 
seniors.  Then,  the  devices  were  placed  at  the  AFHRL  Lackland  Testing  Facility,  Lackland  AFB, 
Texas,  to  test  OTS  pilot  candidates  entering  the  Flight  Screening  Program  (FSP)  from  the  fall  of 
1979  through  the  fall  of  1981.  The  pilot  candidates  were  then  tracked  through  UPT,  and  the 
performance  of  those  who  completed  UPT  between  FY79  and  FY82  was  used  to  validate  the  psychomotor 
test  scores.  All  UPT  student  records  from  FY79  to  FY83  were  obtained  to  ensure  as  many  matching 
cases  as  possible.  The  test  subjects  who  completed  UPT  during  FY83  were  used  to  cross-val idate 
the  developmental  results. 


*.  s 


Concurrent  Validation 


As  an  additional  validation  of  the  psychoaotor  scores,  the  test  devices  were  sent  to  WlHianis 
AFB,  Arizona,  In  June  1984  to  test  UPT  graduating  students.  For  this  sample,  the  results  of  the 
Advanced  Training  Recoa«endat1  on  Board  (ATRB)  were  obtained  as  criteria.  The  ATRB  r  commends 
students  for  follow-on  assignments  to  either  fighter-attack-reconnaissance  (FAR)  or 
tanker-transport-bonber  (TTB)  aircraft,  and  only  the  better  students  receive  a  FAR  recommendation. 

Table  1  shows  the  sample  sizes  of  the  test  groups.  Analyses  reported  here  Include  factor 
analysis  of  the  psychomotor  measures  and  comparisons  among  the  means  for  the  different  training 
outcome  groups  from  UPT.  A  psychomotor  screening  system  and  three  Integrated  selection  systems 
were  developed  using  multiple  linear  regression  techniques.  The  utility  of  the  systems  was 
demonstrated  using  a  hit-miss  classification  algorithm  which  computed  correct  and  Incorrect 
selection  decisions  for  all  of  the  possible  cutoff  scores. 

Table  1.  Subjects  Tested  by  Source 


_ Source _ Subjects  tested 

Air  Force  Academy  382 

Air  Force  Reserve  Officer  Training  Corps  1,229 

Officer  Training  School  603 

Unknown  Source  of  Commission  314 

Testing  Plan  Total  2,528 
UPT,  Williams  AFB  ^ 


_ Total  2,623 

Apparatus 

The  test  devices  administered  both  of  the  psychomotor  tests.  The  internal  computer  generated 
the  test  Images  and  presented  them  on  a  monochrome  cathode-ray  tube  (CRT).  A  pre-recorded 
cassette  tape  played  test  Instructions  to  the  subject  through  headphones.  Two  small  joysticks 
(one  on  the  left  and  the  other  on  the  right  of  the  CRT)  were  used  for  Inputs  during  the  Two-Hand 
Coordination  Test.  An  aircraft-style,  floor-mounted  Joystick  and  two  foot-operated,  rudder-style 
pedals  were  used  for  inputs  during  the  Complex  'Coordination  Test.  The  subject  sat  on  an  ordinary 
chair  In  front  of  the  free-standing  test  device.  Located  on  the  rear  of  the  device  were  three 
digital  displays  that  presented  the  final  scores  of  either  test  that  was  selected  by  a  switch. 
Once  started  by  a  test  administrator,  the  device  would  provide  the  subject  with  5  minutes  of 
Instructions  and  practice  time  for  the  first  test,  then  a  5-m1nute  test  session,  and  then  repeat 
the  same  sequence  for  the  second  test.  Thus  the  entire  procedure  required  only  20  minutes. 


Two-Hand  Coordination  Procedures 

The  first  of  the  tests  -  Two-Hand  Coordination  -  presented  a  triangular-shaped  target  and  a 
cross-shaped  pIpper  on  the  CRT  (Figure  1).  The  computer  moved  the  target  in  an  elliptical  path 
and  with  varying  speeds  (faster  near  the  4  o'clock  position  and  slower  near  the  11  o'clock 
position).  The  subject  moved  the  pIpper  using  the  two  small  joysticks.  The  left  joystick 
controlled  the  pIpper  only  In  the  up-down,  or  vertical,  axis  whereas  the  right  joystick 
controlled  the  pIpper  In  the  left-right,  or  horizontal,  axis.  The  subjects  were  instructed  to 
use  both  Joysticks  simultaneously  In  a  coordinated  manner  to  move  the  pipper,  keeping  it  as  close 
as  possible  to  the  target  on  the  CRT. 
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PSYCHOMOTOR  TESTS 


TEST  I 

•  LEFT  -  RIGHT  HAND  COORDINATION 

•  10  MINUTES 


TARGET 

PIPPER 


I  GRIP 


RIGHT 
I  GRIP 


Figure  1.  Two-Hand  Coordination  Test  Depiction. 


Sixty  times  each  second,  the  computer  measured  In  Inches  the  absolute  distance  from  the 
pIpper  to  the  target  as  both  moved  around  the  CRT.  The  horizontal  and  vertical  components  of 
this  error  measurement  were  computed  and  accumulated  during  the  5-m1nute  test  period.  The  two 
scores  obtained  from  Two-Hand  Coordination  were  the  cumulative  horizontal  (X^)  and  cumulative 
vertical  (Y^)  error  scores. 


Complex  Coordination  Procedures 

The  second  test  -  Complex  Coordination  -  presented  a  set  of  cross-hairs  centered  on  the  CRT, 
a  dot-shaped  pIpper,  and  a  thin  vertical  bar  at  the  bottom  of  the  CRT  (Figure  2).  The  subject 
controlled  the  pIpper,  both  horizontally  and  vertically,  using  the  floor-mounted  joystick.  The 
control  responses  were  the  reverse  of  what  Is  traditionally  required  on  aircraft  (I.e.,  left 
movement  of  the  joystick  moved  the  pIpper  to  the  right,  back  movement  of  the  joystick  moved  the 
pIpper  downward,  etc.).  This  arrangement  was  Intentionally  selected  to  reduce  the  advantage  of  a 
subject  with  prior  flying  experience.  In  the  same  way,  the  vertical  bar  was  moved  horizontally 
to  the  left  by  pressing  the  right  foot  pedal  and  to  the  right  by  pressing  the  left  foot  pedal. 
The  subjects  were  Instructed  to  use  the  joystick  to  center  the  pIpper  horizontally  and  vertically 
on  the  Intersection  of  the  cross-hairs  and  simultaneously  press  the  appropriate  rudder-styl* 
pedal  to  center  the  "rudder  bar"  over  the  lower  part  of  the  vertical  cross-hair.  Ihe  computer 
randomly  changed  the  gain  on  the  Input  controls,  which  drove  the  pIpper  and  rudder  bar  off  their 
marks,  requiring  the  subject  to  constantly  compensate  for  the  forced  displacement. 

Sixty  times  each  second  the  computer  measured  In  Inches  and  accumulated  during  the  5-m1nute 
test  period  the  absolute  distance  from  the  pIpper  to  the  Intersection  of  the  cross-hairs  and  from 
the  rudder  bar  to  the  vertical  cross-hair.  The  three  scores  obtained  from  the  Complex 
Coordination  test  were  the  cumulative  horizontal  error  component  for  the  pIpper  (X2),  the 
cunulatlve  vertical  error  component  for  the  pIpper  (Y2),  and  the  cumulative  horizontal  error 
for  the  rudder  bar  (Z2). 


PSYCHOMOTOR  TESTS 


TESm 

•  STICK  4  RUDDER  SKILLS 

•  10  MINUTES 


Figure  2.  Coaplex  Coordination  Test  Depiction. 
III.  RESULTS 


Psjrchoaotor  Scores 


The  two  psychoaotor  tests  produced  a  total  of  five  error  scores  (X^,  Y] ,  Xg,  Yg, 
Z2)  for  each  subject.  Because  the  scores  measure  error,  low  scores  are  better  than  high 
scores.  Table  2  lists  basic  statistics  for  the  scores  obtained  from  the  original  sample  of  2,528 
candidates  froa  all  commissioning  sources.  The  mean  error  scores  from  the  Two-Hand  Coordination 
Test  were  much  higher  (worse)  than  those  for  the  Complex  Coordination  Test,  which  suggests  that 
the  requirements  for  this  pursuit  tracking  task  were  more  difficult  than  for  the  compensatory 
tracking  task. 

Table  2.  Means,  Standard  Deviations  and  Ranges  of 
Psychoaotor  Scores  (N  -  2,528) 


I 

I 

1 


I 

i 
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Test  score 

Mean 

SD 

Range 

Two-Hand  Coordination 

XI  (Horiz) 

14,709 

5,400 

428  -  65,268 

Y1  (Vert) 

16,748 

5,538 

5,687  -  57,292 

Complex  Coordination 

X2  (Horiz) 

4,514 

6,520 

326  -  71  ,040 

Y2  (Vert) 

3,860 

6,445 

486  -  76,320 

Z2  (Rudder) 

5,580 

6,018 

167  -  71  ,040 

Outliers 


The  values  at  the  high  end  of  the  ranges  of  all  five  scores  were  ouch  larger  than 
anticipated.  Subjects  averaged  approximately  4  Inches  of  tracking  error  for  the  entire  5-m1nute 
test  session  (4  Inches  x  5  minutes  x  60  sec  per  min  x  60Hz  -  72,000  Inches).  These  large  values 
might  be  expected  as  the  error  score  when  the  subject  made  no  tracking  Input  on  that  component  of 
the  test.  However,  none  of  the  subjects  had  extremely  large  values  on  all  five  scores.  If  one 
or  more  axes  were  Ignored,  the  test  taken  by  that  subject  would  be  different  from  the  presented 
test  and  the  scores  would  not  be  comparable  to  the  results  of  other  subjects.  These  extremely 
large  scores  were  from  7.3  to  11.2  standard  deviations  above  the  mean  of  the  respective 
distributions.  There  were  few  such  outliers,  with  the  vast  majority  of  the  scores  within  six 
standard  deviations  of  the  respective  means.  A  cutoff  of  six  standard  deviations  above  the 
respective  mean  was  arbitrarily  Imposed,  removing  35  cases  from  further  analyses.  The  updated 
descriptive  statistics  Including  the  cutoff  of  the  new  sample  of  2,493  subjects  are  listed  In 
Table  3. 


Table  3.  Heans,  Standard  Deviations,  Ranges,  and  Cutoffs 
of  Psychomotor  Scores  (N  ■  2,493) 


Test  Score 

Nean 

SO 

Range 

Cutoff 

Two-Hand  Coordination 

XI  (Horiz) 

14,624 

5,194 

428  -  39,358 

47,109 

Y1  (Vert) 

16,683 

5,427 

5,687  -  45,093 

49,976 

Complex  Coordination 

X2  (Horiz) 

4.113 

4,349 

326  -  42,215 

43,634 

Y2  (Vert) 

3,426 

3,600 

486  -  37,667 

42,530 

22  (Rudder) 

5-502 

.  5,746. . . 

167  -  40.888 

41.688 

Intercorrelatlons 


The  five  test  scores  were  all  measures  of  psychomotor  ability;  therefore,  each  could  Include 
some  common  Information  about  the  subjects.  The  strength  of  this  shared  Information,  expressed 
as  Intercorrelatlons,  Is  shown  In  the  matrix  at  Table  4.  As  might  be  expected,  the  psychomotor 
scores  within  the  same  test  had  the  highest  Intercorrelatlons  whereas  the  correlations  across 
tests  were  small. 


Table  4.  Intercorrelatlons  of  Psychomotor 
Scores  (N  -  2,493) 


Xl 

»1 

XI  (Horiz) 

Y1  (Vert) 

.87 

X2  (Horiz) 

.20 

.25 

Y2  (vert) 

.19 

.23 

.86 

Z2  (Rudder) 

.20 

.24 

.77 

.71 

A  principal  components  factor  analysis  followed  by  a  vaiimax  rotation  further  evaluated  the 
relationships  among  the  five  scores.  The  two  extracted  factors  had  eigenvalues  greater  than  1.0 
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(2.85  and  1.58)  and  accounted  for  88.5%  of  the  original  variance.  Table  5  lists  the  rotated 
factor  loadings.  The  varitnax  rotation  separated  the  scores  clearly  into  two  factors.  Each 
factor  was  dominated  by  only  the  scores  from  a  single  test.  Therefore,  the  two  psychomotor  tests 
were  measuring  different  abilities  with  very  little  in  common  between  the  tests.  The  results  of 
the  factor  analyses  indicate  that  both  of  the  tests  should  be  considered  in  further  validity 
analyses. 


Table  5.  Varimax  Rotated  Factor  Loadings  of 
Psychomotor  Scores  (N  •  2,493) 


Factor  I 


Factor  II 


^1 

Tfl 

^2 

Z2 

10 

.15 

.96 

.88 

.79 

92 

.92 

.11 

.10 

.13 

Criterion  Data 


The  criterion  used  to  assess  the  validity  of  the  psychomotor  test  scores  was  training  outcome 
in  UPT.  Except  for  the  Williams  AFB  sample,  it  was  necessary  to  track  the  tested  candidates 
through  the  49  weeks  of  UPT  before  their  outcome  was  known.  Many  candidates  in  the  sample  failed 
to  start  UPT  because  they  did  not  receive  a  commission  or  were  eliminated  during  normal 
screening.  To  obtain  criterion  data  on  as  large  a  sample  as  possible,  all  of  the  UPT  results 
from  FY79  to  FY83  were  obtained  from  ATC.  Table  6  lists  the  sizes  of  the  validation  samples 
after  matching  the  UPT  criterion  and  psychomotor  data,  A  sample  of  1,725  candidates  who  had 
valid  psychofflotor  test  information  and  ended  UPT  between  FY79  and  FY82  was  identified  and 
analyzed  in  August  1983.  This  group  was  used  for  the  predictive  validation  analyses  and  the 
development  of  the  psychomotor  and  integrated  selection  systems.  A  separately  matched  and  more 
recently  available  sample  of  166  FY83  UPT  students  was  analyzed  in  November  1983  to  cross- 
validate  these  selection  systems.  Finally,  the  results  of  the  95  Williams  AFB  students  tested 
nearing  UPT  graduation  were  analyzed  in  August  1984  for  concurrent  validity  assessment. 


Table  6.  Psychomotor  Validation  Samples 


_ Sample _ 

Predictive  Validation 
(FY79  to  FY82  UPT,  5  bases) 


Cross-Validation 
(FY83,  5  UPT  bases) 


Concurrent  Validation 
(FY84  UPT,  Williams  AFB) 


Predictive  Validation 


For  the  psychomotor  scores  to  be  useful,  at  least  some  of  them  must  significantly 
differentiate  between  candidates  who  graduate  from  UPT  and  those  who  eliminate.  Reasons  for 


elimination  from  UPT  are  classified  as  flying  deficiency,  academic  deficiency,  medical, 

self-initiated,  and  fatality.  If  psychomotor  ability  is  related  to  flying  aptitude,  then  the 
greatest  differences  should  exist  between  the  psychomotor  scores  of  the  graduates  and  those  who 
eliminate  for  flying  deficiency  reasons. 

These  hypotheses  were  evaluated  by  comparing  the  means  of  the  psychomotor  scores  among  three 
categories:  UPT  graduates,  all  UPT  eliminees,  and  UPT  flying  deficiency  eliminees.  Figure  3 

presents  the  differences  among  these  groups.  The  actual  means  with  their  zero-order  correlations 
to  UPT  outcome  and  tests  of  significance  are  listed  in  Appendix  A.  All  five  scores  had 

significant  differences  {£  £.001)  between  graduates  and  either  category  of  eliminees.  In 

addition,  for  all  three  of  the  Complex  Coordination  scores,  the  means  for  flying  deficiency 
eliminees  were  significantly  worse  (p  £.01)  than  those  for  individuals  eliminated  for  all  other 
reasons.  These  results  suggest  that  both  tests  identify  eliminees,  but  the  Complex  Coordination 
Test  does  a  better  job  of  identifying  flying  deficiency  eliminees.  These  results  validate  the 
use  of  all  five  psychomotor  scores  as  predictors  of  success  or  elimination  in  USAF  UPT. 


Concurrent  Validity 

The  psychomotor  testing  of  UPT  students  nearing  graduation  at  Williams  AFB  permitted  the 
concurrent  validation  of  the  tests  against  two  criteria  of  UPT  flying  performance  supplied  by 
ATC.  The  first  was  the  ATRB  decision  whether  or  not  to  recommend  the  student  for  follow-on 
assignment  to  a  fighter-attack-reconnaissance  (FAR)  aircraft.  Graduates  who  were  not 
FAR-recommended  would  receive  a  follow-on  assignment  to  a  tanker-transport-bomber  (TTB) 
aircraft.  Only  the  better  students  received  a  FAR  recommendation.  The  second  criterion  was  an 
experimental  class  ranking  accomplished  by  the  flight  commanders  of  the  UPT  training  squadron. 
Figure  4  presents  all  five  means  for  both  FAR-  and  non-FAR-recommended  students.  The  FAR 
students  had  significantly  (£  £  .01)  better  scores  than  the  non-FAR  students  on  two  of  the  five 
psychomotor  measures  {a  third  score  was  significant  at  ££.05).  Also,  using  multiple  linear 
regression,  the  five  psychomotor  scores  had  significant  (££.01)  multiple  correlations  against 
both  the  class  ranking  (R  =  .41)  and  the  FAR/ncn-FAR  (R  =  .43)  criteria.  These  results  show  that 
in  addition  to  identifying  candidates  with  low  probabilities  of  UPT  success,  psychomotor  scores 
also  relate  to  superior  performance  in  UPT.  Taken  altogether,  these  results  Indicate  that  the 
quality  of  UPT  students  can  be  improved  with  psychomotor  screening. 


Psychomotor  Screening  Equation 


To  obtain  the  maximum  prediction  accuracy  from  the  psychomotor  scores,  a  weighted  equation  or 
linear  model  was  developed  to  predict  UPT  outcome.  This  equation  provides  a  screening  score 
based  on  psychomotor  ability.  Because  the  criterion  was  dichotomous  and  was  coded  0  for 
eliminees  and  1  for  graduates,  the  screening  score  can  be  roughly  interpreted  as  a  probability  of 
success  in  UPT.  To  determine  the  weights  for  the  model,  multiple  linear  regression  was  used, 
with  all  five  psychomotor  scores  as  predictors  and  UPT  outcome  as  the  criterion  variable. 

Although  all  five  psychomotor  scores  were  significant  predictors  of  UPT  outcome,  the  scores 
of  each  test  were  highly  interrelated,  as  shown  by  the  factor  analyses.  Accordingly,  the  most 
useful  linear  model  would  contain  the  fewest  psychomotor  scores  which  still  accounted  for  as  much 
of  the  criterion  variance  as  all  five  scores  together.  However,  that  does  not  mean  that  the 
tests  could  be  changed  to  present  only  the  most  predictive  axes,  because  that  would  change  the 
nature  of  the  tasks  in  the  tests.  After  several  comparisons  using  the  F-ratio,  the  final 


Figure  4.  Mean  Psycho«otor  Scores  by  FAR/Mon-FAR  (TIB). 

(N  -  95) 


screening  equation  (R  =  .196)  contained  only  the  and  Y2  scores.  (Appendix  B  sunnarlzes 
this  analysis.)  Therefore,  the  most  useful  screening  score  based  on  psychomotor  ability  can  be 
computed  from  the  following  equation: 


Y.j  =  a  +  (b]  X  Xl-j)  +  (b2  x  Y2'() 


where  Y^ 
a 

h 

b2 

XI1 

Y2i 


=  psychomotor  screening  score  of  1th  subject 
=  constant  (0.95648) 

*  weight  of  psychomotor  Xi  score  (-0.0000075748) 

*  weight  of  psychomotor  Y2  score  (-0.000020375) 

=  psychorootor  X^  score  of  1th  subject 

=  psychomotor  Y2  score  of  1th  subject 


This  psychomotor  screening  score  was  computed  for  the  1,725  cases  In  the  developmental 
sample.  The  mean  screening  scores  for  the  UPT  graduates  (79.0)  and  UPT  ellmlnees  (75.1)  are 
depicted  In  Figure  5.  Though  the  difference  In  the  means  was  small.  It  was  statistically 
significant  (^£.001). 


Psychomotor  Screening  System 

The  next  step  In  the  analyses  was  to  determine  the  practical  value  of  these  scores 
differences.  To  use  this  Information  In  a  selection  system,  the  psychomotor  screening  scores 
could  be  computed  from  the  psychomotor  test  results  for  all  available  candidates.  The  candidates 
could  then  be  rank-ordered,  based  on  the  computed  screening  scores;  and  the  best  candidates  could 
be  selected  to  fill  the  required  training  quotas.  Figure  6  depicts  such  a  ranking  of  the  1,725 
subjects  of  the  development  sample.  The  sample  was  segmented  Into  decile  groups  (lOK  each)  for 
ease  of  comparison.  The  lowest-ranked  decile  Is  on  the  far  left  and  the  highest-ranked  is  on  the 
far  right.  Each  decile  bar  was  further  divided  by  the  percentage  of  that  decile  that  actually 
eliminated  from  UPT.  As  shown  by  the  trend  across  the  deciles,  the  lower-ranked  deciles  had 
greater  percentages  of  ellmlnees  than  did  the  deciles  ranked  higher  by  the  psychomotor  screening 
system. 


Utilization  of  Psychomotor  Screening  System 

If  all  available  candidates  could  be  compared  simultaneously,  then  the  selection  procedure 
could  simply  choose  the  number  of  candidates  required  from  those  with  the  highest  screening 
scores.  In  operational  use,  this  procedure  would  be  complicated  by  the  problems  of  multiple 
commissioning  sources  and  the  long-term  planning  required  In  the  UPT  selection  process.  An 
alternate  approach  would  select  only  those  candidates  who  obtain  a  screening  score  above  a 
predetermined  cutoff.  The  cutoff  score  would  be  determined  and  updated  by  comparing  the 
recruiting  environment  and  UPT  attrition  Information.  This  approach  would  permit  psychomotor 
screening  to  function  as  a  "gate"  at  any  step  In  the  selection  process  In  the  same  way  that  the 
AFOOT  and  FSP  results  are  currently  used  (I.e.,  candidates  with  scores  above  the  minimum  cutoff 
proceed  to  the  next  phase  of  selection  while  those  below  the  cutoff  are  rejected). 


Psychoaotor  Screening  Score  by  UPT  Outcow 
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Figure  6.  Decile  Groups  Ranked  by  Psychoawtor  Screening  Score. 


Th€  higher  the  psychonotor  screening  cutoff  score  is  set,  the  greater  the  inpact  on  attrition 
in  UPT.  Hotfever,  some  rejection  of  candidates  who  could  pass  training  will  also  occur.  This 
•false  rejection*  is  a  problen  with  all  screening  systens,  including  flying  screening,  and  must 
be  evaluated  in  terns  of  the  percentages  screened  and  the  quality  of  the  graduating  students. 
Since  the  psychOMtor  scores  were  related  to  neasures  of  UPT  quality  (class  ranking  and  FAR 
recoaaendation),  it  is  likely  that  the  potential  graduates  in  the  lower  deciles  who  would  be 
"false  rejections*  would  be  poorer-perforaing  graduates  who  would  require  more  instruction  and 
flying  time. 

Table  7  shows  some  examples  of  cutoff  scores  with  the  corresponding  results  that  would  have 
occurred  had  the  cutoff  applied  to  the  1,725  cases  of  the  development  sample.  For  each  cutoff 
score,  the  hit-miss  classification  provided  the  percentile  of  the  entire  sample  which  would  be 
screened,  the  percentage  of  actual  UPT  attritees  in  the  sample  who  would  be  correctly  screened 
and  the  percentage  of  UPT  graduates  in  the  sample  who  would  be  falsely  rejected. 


Table  7.  Examples  of  Psychomotor  Screening 


Cutoff  score 
percentile 

S  of  attritees 
correctly  rejected 

X  of  graduates 
incorrectly  rejected 

5th 

11.1 

3.5 

10th 

19.6 

8.0 

20th 

31.8 

15.7 

30th 

45.9 

26.7 

Cross-Velidation 


Results  were  cross-val (dated  using  166  candidates  who  completed  UPT  in  FY83.  These  were  the 
latest  data  from  UPT  available  at  the  time  of  the  analyses.  The  screening  scores,  computed  for 
the  FY83  sample  using  the  psychomotor  screening  equation  above,  correlated  significantly 
(£^£.005)  with  UPT  outcome  (R  ■  .23).  The  correlation  was  slightly  larger  than  those  found  in 
the  development  samples,  but  this  sample  was  considerably  smaller.  These  results  validate  the 
psychomotor  screening  system  on  a  more  recent  sample  that  was  independent  of  its  development. 
This  suggests  that  psychomotor  measures  are  sufficiently  robust  to  be  collected  several  years 
prior  to  UPT  and  still  permit  the  psychomotor  screening  system  to  be  effectively  applied  at  UPT 
entry. 


IV.  INHCRATEO  SELECTION  SYSTEM 

A  potentially  more  effective  approach  to  pilot  candidate  selection  would  consider  all  valid 
screening  information  simultaneously,  including  psychomotor.  The  advantage  of  an  integrated 
approach  is  that,  through  analytical  techniques,  the  best  mix  of  the  information  from  the 
multiple  sources  can  be  obtained,  with  components  selectively  weighted  to  increase  screening 
effectiveness.  Such  an  Integrated  system  would  contain  a  selection  profile  with  the  ranges  of 
all  the  screening  measures  known  for  UPT  graduates.  A  candidate's  scores  on  all  of  the  screening 
measures  would  be  simultaneously  compared  to  the  successful  graduate  profile  to  produce  an 
overall  predicted  probability  of  UPT  success. 


Current  operational  USAF  pilot  selection  considers  information  from  the  five  composite  scores 
of  the  AF0()T,  age  at  the  start  of  UPT,  possession  of  a  college  technical  degree,  and  possession 
of  a  private  pilot's  license  (PPL)  or  completion  of  a  USAF  light  aircraft  flying  program  like  the 


FSP.  Upon  investigation,  USAF  historical  records  were  found  to  be  very  liaited  regarding  the 
possession  of  a  PPL  and  the  flying  results  of  the  AFA  Pilot  Indoctrination  Prograa  (PIP)  and  the 
AFROTC  Flight  Instruction  Program  (FIP).  Therefore,  these  data  were  not  included  in  the 
developmental  analyses.  The  only  flying  information  included  was  the  standardized  results  of  the 
FSP  conducted  at  Hondo  AFB,  Texas.  The  technical  degree  information  and  FSP  final  outcome 
information  were  used  as  dichotomous  variables  and  coded  1  or  0  as  appropriate.  Two  continuous 
variables  based  on  FSP  results  were  also  included.  These  were  the  grade  of  the  final  evaluation 
flight  and  the  sum  of  the  grades  from  all  of  the  other  flights.  In  addition,  gender  and  race  of 
the  candidates  were  included  in  the  analysis.  Appendix  C  lists  the  zero-order  correlations  of 
these  measures  with  the  criterion  (UPT  graduation/elimination  for  all  reasons),  each  derived  for 
all  available  subjects  from  the  historical  (FY79  to  FY83)  records. 

Model  Development 


Because  pilot  candidates  are  chosen  differently,  depending  on  their  source  of  commission,  the 
available  sets  of  historical  information  were  different  for  each  commissioning  source. 
Therefore,  three  Integrated  Pilot  Candidate  Selection  Models  (IPCSMs)  were  developed. 
Specifically,  the  FSP  results  were  available  only  for  OTS  candidates.  In  addition,  because  AFA 
candidates  were  not  required  to  take  the  AFOQT,  these  scores  were  available  only  for  OTS  and 
AFROTC  candidates.  Each  IPCSM  was  developed  from  the  psychomotor  scores  and  the  specific 
information  currently  used  to  select  candidates  from  each  major  commissioning  source.  The 
samples  used  to  develop  each  IPCSM  are  presented  in  Table  8.  As  was  done  for  the  psychomotor 
screening  system,  the  three  IPCSMs  were  reduced  to  the  smallest  set  of  measures  possible  without 
losing  significant  prediction,  using  F-ratios  to  compare  the  results  of  the  various  multiple 
linear  regression  analyses. 


Table  8.  Sample  Sizes  for  Integrated 
Pilot  Candidate  Selection  Models 


IPCSMs 

Sample  Size 

IPCSM  I  (OTS  Model) 

268 

IPCSM  II  (AFROTC  Model) 

753 

IPCSM  III  (AFA  Model) 

310 

IPCSM  I  (OTS  Model).  The  268  subjects  for  IPCSM  I  (OTS  Model)  had  17  measures  available 

(five  psychomotor  scores,  age,  five  AFOQT  composite  scores,  three  FSP  results,  technical  degree, 

gender,  and  race).  This  full  model  was  reduced  in  a  logical  stepwise  fashion  down  to  a  final 
seven-variable  model.  The  F-ratio  between  the  full  model  and  the  seven-variable  final  model 
(containing  psychomotor  Xf  and  Y2  scores,  age,  AFOQT  pilot  and  navigator  composite  scores, 
and  FSP  sum  of  lesson  grades  and  final  grade)  revealed  no  significant  difference  in  prediction. 
Appendix  0  contains  a  summary  of  the  analyses.  In  the  presence  of  these  measures,  gender,  race, 
and  possession  of  a  technical  degree  did  not  add  to  the  prediction  of  UPT  outcome. 

The  final  prediction  measures  of  IPCSM  I  were  regressed  against  UPT  outcome  to  produce  a 

profile  of  a  UPT  graduate  in  the  form  of  a  weighted  equation.  IPCSM  I  (OTS  Model)  will  produce  a 

score  corresponding  to  the  probability  of  UPT  success  using  the  following  equation: 

Yj  «  a  ^  (b]  X  X1{ )  'f  (b2  X  Y2^ )  ♦  (b3  x  age^ ) 

+  (b4  X  AFOQTPf )  (bs  X  AFOQTN^ ) 

♦  (bg  X  FSP1<)  ♦  (by  X  FSP2^) 
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where  =  IPCSM  I  Score  for  ith  subject 

a  »  constant  (0.10192) 

b]  »  weight  for  psychooiotor  XI  score  (-.000015211) 

bp  =  weight  for  psychoontor  Y2  score  (-.000010618) 

b3  =  weight  for  age  of  subject  (-.0012) 

b4  “  weight  for  AFOQTP/AFOQT  pilot  score  (0.00112) 

b5  =  weight  for  AF0QTN/AF0()T  navigator  score  (0.00029067) 

65  =  weight  for  FSP1/FSP  final  grade  (0.08249) 

bj  =  weight  for  FSP2/FSP  sum  of  grades  (0.01923) 

XI  ^  =  psychoinotor  XI  score  of  1th  subject 

Y2^  *  psychofliotor  Y2  score  of  1th  subject 

age^  =  age  at  start  of  UPT  of  1th  subject 

AFOQTP^  =  AFOQT  pilot  composite  score  of  1th  subject 

AFOQTM^  =  AFOQT  navigator  composite  score  of  1th  subject 

FSPl^  =  FSP  final  grade  of  1th  subject 

FSP2^  =  FSP  sum  of  grades  of  1th  subject 

IPCSH  II  (AFROTC  Model).  Similar  analyses  were  performed  for  IPCSM  II  (AFROTC  Model)  using 
753  subjects  who  had  14  measures  available  (five  psychomotor  scores,  age,  five  AFOQT  composite 
scores,  technical  degree,  gender,  and  race).  The  F-ratIo  between  the  full  model  and  a  final 
five-variable  model  (psychomotor  X]  and  Yp  scores,  age,  and  AFOQT  pilot  and  navigator 
composite  scores)  revealed  no  significant  difference  In  prediction.  Appendix  D  provides  a 
summary  of  the  analyses.  Again,  technical  degree,  gender,  and  race  were  found  to  be  redundant  In 
the  presence  of  psychomotor,  age,  and  AFOQT  Information. 

The  final  prediction  measures  of  IPCSM  tl  were  regressed  against  UPT  outcome  to  produce  a 
weighted  profile  equation.  IPCSM  II  will  produce  a  score  corresponding  to  the  probability  of  UPT 
success  score  using  the  following  equation: 

Y|  -  a  +  (bi  X  X1^)  +  (bp  x  Y2^)  +  (b3  x  age^) 

+  (b4  X  AFOQTP^)  +  (b5  x  AFOQTM^) 

where  Y^  «  IPCSM  II  Score  for  1th  subject 

a  ■  constant  (1.72485) 

bi  •  weight  for  psychomotor  XI  score  (-.0000030409) 

bp  weight  for  psychomotor  Y2  score  (-.000022526) 

63  ■  weight  for  age  of  subject  (-.04517) 

b4  -  weight  for  AFOQTP/AFOQT  pilot  score  (0.00124) 

65  ■  weight  for  AFOQTN/AFOQT  navigator  score  (0.00225) 

The  common  variables  are  as  defined  In  the  IPCSM  I  equation. 

IPCSM  III  (AFA  Model).  Finally,  the  310  subjects  for  IPCSM  III  (AFA  Model)  had  eight 
measures  available  (five  psychomotor,  age,  technical  degree,  and  race).  Gender  was  not  Included 
because  a.  ^FA  candidates  In  this  sample  were  males.  The  F-ratIo  between  the  full  model  and  a 
final  four- <ar1ab1e  restricted  model  (psychomotor  X^  and  Yp  scores,  age,  and  technical 
degree)  revealed  no  significant  difference  In  prediction.  Appendix  0  shows  a  sumnary  of  these 
analyses.  Without  AFOQT  d;:ta  available,  technical  degree  was  found  to  be  a  significant  predictor 
In  IPCSM  III  (AFA  Model). 

The  final  prediction  measures  of  IPCSM  III  were  regressed  against  UPT  outcome  to  produce  a 
weighted  profile  equation.  IPCSM  III  will  produce  a  score  corresponding  to  the  probability  of 
UPT  success  score  using  the  following  equation: 


a  +  (b-i  X  Xli)  +  (b2  X  Y2^ )  +  (b3  x  age,  )  +  (b4  x  tech,  ) 


Yi  = 
where 

a 

b) 

bg 

^3 

b4 

tech,- 


IPCSM  III  Score  of  ith  subject 
constant  (1.43056) 

weight  for  psychomotor  XI  score  (-.0000050818) 
weight  for  psychomotor  Y2  score  (-.0000072309) 
weight  for  age  of  subject  (-.02111) 
weight  for  possession  of  technical  degree  (.08756) 

1  if  ith  subject  has  a  technical  degree,  0  otherwise 


Common  variables  are  as  defined  in  the  IPCSM  I  equation. 


Predictive  Validation 


The  weighted  equations  of  the  three  IPCSMs  were  used  to  generate  integrated  selection  scores 
for  the  pilot  candidates  of  the  three  development  samples  within  each  IPCSM  classification. 
These  selection  scores  correspond  to  a  candidate's  probability  of  UPT  success  as  predicted  by  the 
measures  in  IPCSM.  The  differences  of  the  mean  predicted  scores  between  UPT  graduates  and 
eliminees  for  all  three  IPCSMs.  and  the  correlations  between  these  scores  and  UPT  outcome,  were 
significant  (££.01)  and  are  shown  by  Figure  7.  These  results  validate  the  use  of  all  three 
IPCSMs  as  predictors  of  success  or  elimination  in  USAF  UPT. 


Utilization  of  the  IPCSM  Systems 


As  was  done  for  the  psychomotor  screening  system,  the  practical  value  of  the  significant 
differences  in  scores  between  graduates  and  eliminees  was  demonstrated  for  the  three  samples  by 
separately  ranking  them  on  their  IPCSM  scores.  For  example,  the  268  subjects  in  the  IPCSM  I  (OTS 
Model)  sample  were  rank-ordered  and  divided  into  deciles  as  depicted  in  Figure  8.  Again,  the 
lowest-ranked  decile  is  on  the  far  left,  the  highest-ranked  is  on  the  far  right,  and  the  bottom 
part  of  each  decile  bar  is  further  divided  by  the  percentage  of  that  decile  that  actually 
eliminated  from  UPT. 

The  trend  toward  increasing  percentages  of  eliminees  within  each  higher-ranked  decile  is 
steeper  in  IPCSM  I  than  for  the  psychomotor-only  screening  model  (Figure  6)  because  of  the 
increased  accuracy  obtained  by  integrating  the  selection  information.  Though  not  presented,  the 
corresponding  plots  for  IPCSMs  II  and  III  were  similar. 

Any  of  the  three  IPCSMs  could  be  used  to  select  UPT  candidates  in  the  same  manner  suggested 
for  the  psychoraotor  screening  system.  A  predetermined  cutoff  score  would  form  the  screening 
point.  Tables  9,  10,  and  11  list  examples  of  cutoff  scores  for  the  three  IPCSMs  with  the  results 
that  would  have  occurred  had  the  cutoff  applied  to  the  corresponding  cases  of  the  development 
samples.  For  each  cutoff  score,  the  hit-miss  classification  provided  the  percentile  of  the  total 
sample  which  would  have  been  screened,  and  the  percentage  of  actual  UPT  attritees  and  percentage 
of  UPT  graduates  at  or  below  the  cutoff.  Each  IPCSM  would  have  correctly  screened  the  attritees 
and  incorrectly  rejected  the  potential  graduates  by  the  indicated  percentages. 

Both  the  percentages  of  eliminees  correctly  screened  and  potential  graduates  who  would  be 
incorrectly  screened  showed  Improvement  over  the  psychomotor  screening  system  at  every  selection 
score  cutoff  point  in  these  examples.  Each  IPCSM  system,  because  of  the  integration  of  all  of 
the  available  information,  appears  to  identify  more  eliminees  and  screen  fewer  potential 
graduates  than  use  of  the  same  information  in  a  series  of  gates. 
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Mean  IPCSN  Scores  by  UPT  Outcome. 
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Figure  8.  Decile  Groups  Ranked  by  IPCSM  I  (OTS  Model). 

(N  -  268} 


Table  9.  Examples  of  IPCSM  I  (OTS  Model)  Screening 


Cutoff  score 
percentile 

%  of  attrltees 
correctly  rejected 

X  of  graduates 
Incorrectly  rejected 

5th 

18.4 

0.0 

10th 

27.6 

2.6 

20th 

47.4 

10.4 

30th 

57.9 

19.3 

Table  10. 

Examples  of  IPCSM  H 

(AFROTC  Model)  Screening 

Cutoff  score 

X  of  attrltees 

X  of  graduates 

percentile 

correctly  rejected 

Incorrectly  rejected 

5th 

15.0 

2.5 

10th 

27.2 

6.6 

20th 

38.8 

15.7 

30th 

54.4 

26.2 

Table  11. 

Examples  of  IPCSM  III 

(AFA  Model)  Screening 

Cutoff  score 

X  of  attrltees 

X  of  graduates 

percentile 

correctly  rejected 

Incorrectly  rejected 

5th 

16.1 

4.3 

10th 

32.3 

8.6 

20th 

48.4 

15.8 

30tb 

51.6 

26.2 

Cross-Validation 


The  FY83  DPT  sample  was  again  used  to  cross-valldate  the  three  IPCSMs.  IPCSM  I  Integrated 
scores  were  computed  for  the  82  subjects  with  all  seven  measures  available  and  then  correlated  (R 
■  .33)  with  UPT  outcome.  Though  some  shrinkage  In  correlation  occurred  for  IPCSM  I,  the 
relationships  were  still  significant  (^^.001).  For  IPCSM  II,  64  subjects  had  five  measures 
available  which  significantly  correlated  (R  »  .33,  £  i  .005)  with  UPT  outcome.  Thus,  IPCSMs  I 
and  II  cross-validated  as  predictors  of  UPT  outcome  on  a  more  recent  sample  which  was  Independent 
of  their  development.  Finally,  165  subjects  had  the  four  measures  of  IPCSM  Hi  which  correlated 
(R  ■  .11)  with  UPT  outcome.  This  correlation  was  not  significant  (£  £  .07),  though  the 
correlation  In  the  development  sample  was;  and  questions  about  the  general  1  zabi  1  Ity  of  IPCSM  HI 
remain  unanswered. 


V.  CONCLUSIONS/RECOMMENDATIONS 

The  results  of  the  analyses  in  this  report  demonstrate  that  the  two  psychomotor  tests 
generated  by  the  computer-based  test  devices,  Two-Hand  Coordination  and  Complex  Coordination, 
produce  scores  that  are  valid  predictors  of  UPT  outcome.  Candidates  who  are  likely  to  graduate 
from  UPT  can  be  differentiated  by  their  scores  from  those  who  are  likely  to  eliminate.  Also, 
superior  UPT  students  (FAR-recommended)  can  be  differentiated  from  weaker  students  (non-FAR). 

The  differentiation  of  pilot  candidates  could  be  obtained  through  the  Implementation  of  the 
psychomotor-only  screening  system  In  the  fashion  of  other  screening  gates  currently  being  used 


20 


(e.g.,  AFOQT,  FSP).  Since  the  cross-validation  results  demonstrated  the  robustness  of  the 
psychomotor  measures  over  time,  psychomotor  testing  could  be  administered  very  early  in  the 
selection  process  and  still  significantly  predict  the  individual's  probability  of  success  without 
the  need  to  retest  the  candidate  at  UPT  entry.  The  strength  of  prediction  and  costs  of 
psychomotor  testing  would  be  expected  to  be  greater  than  those  associated  with  the  administration 
of  the  AFOQT  but  less  than  for  flying  screening.  This  could  determine  the  logical  sequence  for 
these  three  screening  gates.  To  maximize  the  impact  on  attrition  and  the  quality  of  the  students 
selected  for  UPT,  the  highest  possible  psychomotor  screening  scores  should  be  selected  as  the 
cutoffs.  This  policy  would  screen  the  largest  percentage  of  the  candidates  who  would  attrite. 
However,  many  potential  graduates  would  also  be  rejected. 

Psychomotor  scores  are  not  the  only  important  information  which  should  be  used  In  the 
selection  of  pilots;  rather,  all  significant  predictors  should  be  utilized.  The  IPCSMs 
capitalize  on  the  utility  of  all  available  information.  Including  psychomotor  measures,  to 
predict  success  or  failure  in  UPT  as  well  as  quality  of  performance  as  measured  by  FAR 
recommendations.  The  "Whole  Person  Concept"  (an  appraisal  of  the  important  strengths  of  a 
candidate)  is  Imbedded  In  this  multi-dimensional  IPCSM  approach  to  pilot  selection.  The  IPCSMs 
will  not  only  impact  attrition  and  quality  in  UPT,  but  will  also  reduce  the  rejection  of 
potential  graduates.  Also,  IPCSMs  I  and  11  can  be  used  to  screen  minorities  and  women  without 
consideration  of  race  or  gender. 

Although  the  samples  used  in  the  development  of  the  IPCSMs  were  aligned  with  a  particular 
source  of  commission,  that  restriction  was  data  dependent.  The  cross-validation  of  IPCSMs  I  and 
II  were  not  restricted  by  source  of  commission.  Therefore,  the  choice  of  the  appropriate 
selection  model  to  use  should  depend  on  the  information  available.  IPCSM  I  is  the  strongest 
system  (highest  correlation  with  UPT  outcome)  and  is  recommended  for  use  as  the  final  selection 
system  for  candidates  going  through  FSP  (traditionally  OTS  cadets  and  recently,  AFROTC  cadets 
attending  Summer  Field  Training  at  Lackland  AFB,  Texas).  IPCSM  II  could  be  used  as  a 
preselection  input  either  for  OTS  or  for  AFROTC  Field  Training. 

Unfortunately,  IPCSM  III  does  not  appear  sufficiently  stronger  in  prediction  than  the 
psychomotor  screening  system  alone  and  therefore  does  not  warrant  implementation  in  its  current 
form.  However,  if  objective  measures  similar  to  the  FSP  and  AFOQT  information  were  available  for 
AFA  candidates  in  addition  to  the  currently  available  data  and  psychomotor  scores,  then  results 
similar  to  those  from  IPCSM  I  could  be  expected.  A  follow-on  RSD  effort  to  obtain  these  measures 
for  Acadeiay  cadets  and  validate  an  IPCSM  system  for  them  Is  recommended. 

Though  the  systems  presented  in  this  report  will  Improve  the  selection  of  candidates  for  UPT, 
further  refinements  are  possible.  One  improvement  would  develop  the  weights  of  the  selection 
equations,  not  using  the  dichotomous  UPT  pass/fail  criterion  but  Instead,  using  a  continuous 
measure  of  the  individual's  flying  performance  both  in  training  and  on  operational  missions.  The 
candidates  thus  selected  for  training  would  have  good  chances  of  becoming  successful  operational 
pilots.  The  data  to  produce  a  Pilot  Performance  Index  to  provide  this  enhancement  are  being 
collected  by  AFHRL,  and  these  analyses  are  planned  for  the  near  future.  Another  Improvement  to 
these  systems  would  include  information  on  candidates  who  possess  a  PPL.  Research  which  compares 
the  psychomotor  test  performance  of  FSP  graduates  and  candidates  with  PPLs  is  recommended  to 
extend  the  general izability  of  these  selection  systems. 

The  addition  of  psychomotor  screening  to  the  selection  procedure  does  not  preclude  the  search 
for  other  valid  measures  of  flying  aptitude.  Every  valid  measure  added  to  an  integrated 
selection  procedure  would  help  reduce  the  attrition  and  false  rejections  in  training.  A 
promising  research  area  is  the  measurement  of  mental  aptitudes  such  as  information  processing  and 
decisiveness.  The  advent  of  extremely  powerful  microcomputers  makes  possible  the  measurement  of 
reaction  time  and  accuracy  on  very  complex  mental  tests.  Including  such  validated  measures  in 
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the  IPCSM  systems  would  Improve  the  current  results.  AFHRL  has  developed  a  battery  of  mental  and 
personality  tests,  called  the  Basic  Attributes  Tests  (BAT),  for  this  expressed  purpose.  These  13 
tests  (Imhoff  A  Levine,  1981)  are  administered  by  a  highly  portable  and  flexible  test  device, 
called  the  Porta-BAT.  Results  of  this  ongoing  effort  will  be  reported  In  the  future. 

In  conclusion,  the  Implementation  of  psychomotor  screening,  either  as  a  separate  gate  or 
within  an  Integrated  selection  system,  will  Improve  the  quality  of  student  candidates  In  USAF 
UPT.  Based  on  the  results  of  this  current  effort,  an  Integrated  selection  system  Including 
psychomotor  screening  Is  recoomended.  Future  research  should  extend  both  the  selection 
information  being  considered  and  the  criteria  to  be  predicted. 
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LIST  OF  ABBREVIATIONS 


AFA 

AFB 

AFHRL 

AFOQT 

AFROTC 

AFSC 

ATC 

ATRB 

BAT 

CRT 

FAR 

FIP 

FSP 

IPCSH 

OTS 

PIP 

PPL 

RPR 

RAD 

TTB 

UPT 

USAF 


Air  Force  Acadeuiy 
Air  Force  Base 

Air  Force  Human  Resources  Laboratory  (AFSC) 
Air  Force  Officer  Qualifying  Test 
Air  Force  Reserve  Officer  Training  Corps 
Air  Force  Systems  Command 
Air  Training  Connand 

Advanced  Training  Recommendation  Board  (ATC) 
Basic  Attributes  Tests 
Cathode-Ray  Tube 

Fighter-attack-reconnaissance  (training  track) 
Flight  Instruction  Program  (AFROTC) 

Flight  Screening  Program  (OTS) 

Integrated  Pilot  Candidate  Selection  Model 

Officer  Training  School 

Pilot  Indoctrination  Program  (AFA) 

Private  Pilot's  License 
Request  for  Personnel  Research 
Research  and  Development 
Tanker-transport-bomber  (training  track) 
Undergraduate  Pilot  Training 
United  States  Air  Force 
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APPEWDIX  A;  COMPARISON  OF  PSYCHOMOTOR  SCORES  BY  UPT  OUTCOME 


Psvchoaotor  Scores 

X] 

^2 

l2 

Correlation 

with  UPT  outcone(N  »  1,725) 

-.12 

-.10 

-.14 

-.17 

-.15 

Means  by  UPT  Outcome^ 

Graduate  Means (N  >  1,348) 

14,315 

16.341 

3.559 

2.858 

4.725 

All  El  1m  Means(N  377) 

15,829 

17.621 

4.936 

4.173 

6.670 

Flying  Deficiency  (FD) 

El  1m  Means  (N  -  199) 

16.302 

18.007 

5.593 

4.702 

7.580 

Probabilities  of  Psychomotor  Score  Differences  Occurring  by  Chance  Alone 

Grads  vs.  All  El  1ms 

.001 

.001 

.001 

.001 

.001 

Grads  vs.  FD  El  1ms 

.001 

.001 

.001 

.001 

.001 

FO  vs.  Other  El  1ms 

.103 

.183 

.006 

.010 

.005 

•Psycho«otor  scores  reflect  errors;  therefore,  lower  scores  mean  better  test 
performance. 
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WEMDIX  B;  PSVCNONOTOR  SCREENING  MODEL  REDUCTION 


Full  Model 
Restricted  Model 


Psychoaotor 

variables 


X1,Y1,X2,Y2,Z2 


F>«ATI0  TEST  (N  -  1.725) 


Independent 
redictors 


6  .04225 


value* 


XI,  Y2 


.03842 


(R?  -  r|)  /  (IPi  -  IP?) 


(  1  -  RZ  ,  )  /  (  N  -  IP,  ) 


F  (-0*225  -  .03827)  /  (6  -  3) 

(  1  -  .04225  )  /  (  1725  -  6  ) 


F  •  2.38 


Mote.  No  significant  difference  In  prediction  strength  of  these  two 
■odels  since  coaqiuted  F*rat1o  value  does  not  exceed  the  critical  F  value 
(Fe  ■  3.78). 

*Probab111ty  of  R^  Occurring  by  Chance  Alone. 
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APPENDIX  C;  ZERO-ORDER  CORRELATIONS  IPCSM  PREDICTORS  WITH  UPT  OUTCOME 


Predictors 

Correlation  with  UPT 
oass/fall 

M* 

FSP  Pass/Fall 

.137 

1534 

FSP  Final  Grade 

.271 

1534 

FSP  Sum  of  Grades  2-11 

.303 

1534 

AFOQT  Pilot  score 

.158 

4460 

AFOQT  Navigator  Technical 

.148 

4460 

score 

AFOQT  Academic  score 

.080 

4577 

AFOQT  Verbal  score 

.007 

4576 

AFOQT  Quantitative  score 

.138 

4577 

Psychofflotor  XI  score 

-.118 

1918 

Psychomotor  Y1  score 

-.099 

1919 

Psychomotor  X2  score 

-.153 

1906 

Psychofflotor  Y2  score 

-.181 

1910 

Psychomotor  Z2  score 

-.146 

1916 

Age 

-.120 

8438 

Gender 

.033 

8438 

Race 

.110 

8292 

Possession  of  Tech  Oeoree 

.111 

8183 

% 
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APPENDIX  D:  lPCS^■.s  REDUCTION  USING  F-RATIO  TESTS 


Full  model 


Restricted  Model 


IPCSM  I  Variables 


Psychomotor{5) 

FSP{3) 

AFOQT(E) 

Age 

Tech  Degree 

Race 

Gender 


Ps>choniotor(2) 

FSP(2) 

AFOQT(2) 

Age 


Analyses  results 


r2  -  .23254 
df  «  17 
N  «  265 
£_  <  .  0000 


r2  =  ,21501 
df  =  7 
N  =  265 
£  1 . 0000 


IPCSM  II  variables 


Psychomotor(5) 

AF0QT(5) 

Age 

Tech  Degree 

Race 

Gender 


Psychomotor  (2) 
AF0Q1(2) 

Age 


Analyses  results 


r2  -  .09673 
df  -  15 
N  «  741 
2^  <_ .  0000 


r2  =  .08437 


df  =  6 
N  =  741 
2  1  . 0000 


IPCSM  III  variables 


PsychomotortS) 
Tech  Degree 
Age 
Race 


Psychomotor (2) 
Tech  Degree 
Age 


Analyses  results 


r2  -  .09673 
df  -  15 
N  -  741 

D  <  .0000 


r2  .  .08437 
df  =  6 
N  «  741 
2  1 . 0000 


(R^  1  -  g)  /  (IPl  -  IP2) 
(1  -  R2  l)  /  (N  -  IPl) 


F-ratIo  test  calculations 


.  (.23254  -  .21501)  /  (18  -  8) 


(1  -  .23254)  /  (265  -  18) 


F  «  0.56® 


F(cr1t1cal )  =  2.41 


F-ratIo  test  calculations 


F  =  (.09673  -  .08437)  /  (15  -  6) 
(1  -  .09673)  /  (741  -  15) 


F(crltlcal)  «  2.41 


F-ratIo  test  calculations 


P  .  (.05581  -  .04294)  /  (9  -  5) 
(1  -  .05581)  /  (305  -  9) 


F(crltlcal)  =■  3.32 


*Mo  significant  difference  In  models.*  u.s*  covernmf.nt  PRrNTiNC  officf:  1986- e  s  9  -  o  s  s  /  4  7o?a 
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